Constructing ensembles of classifiers using linear projections based on misclassified instances

In this paper we propose a novel approach for ensemble construction based on the use of linear projections to achieve both accuracy and diversity of individual classifiers. The proposed approach uses the philosophy of boosting, putting more effort on difficult instances, but instead of learning the classifier on a biased distribution of the training set it uses misclassified instances to find a linear projection that favours their correct classification. Supervised linear projections are used to find the most suitable projection at each step of the creation of the ensemble. In a previous work we validated this approach using non-linear projections. In this work we show that linear projections can be used as well, with the advantage of being simpler, more interpretable and faster to obtain. The method is compared with AdaBoost, showing an improved performance on a large set of 45 problems from the UCI Machine Learning Repository.